{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [基线方法](https://work.datafountain.cn/forum?id=564&type=2)\n",
    "任务描述：建立一个预测模型，通过从序列中提取特征，对microRNA和gene的序列关系进行预测数据集:microRNA的序列信息和gene信息已经提供，并且提供了一部分的关系对信息，需要使用这些关系对信息建模，预测测试集中的microRNA和gene序列是否有关系方法概述：首先从序列中提取3-mer的信息并且计数，获取microRNA和gene的3-mer的分布信息，再将二者拼接做成数据集，使用随机森林进行预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import itertools\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import roc_auc_score\n",
    "import os\n",
    "import joblib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.数据准备\n",
    "\n",
    "1.1 数据说明\n",
    "\n",
    "1.2 数据预处理\n",
    "\n",
    "1.3 数据集生成"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#数据说明\n",
    "dataset=pd.read_csv('./datasets/train_dataset/Train.csv')\n",
    "mirna_seqdf=pd.read_csv('./datasets/train_dataset/mirna_seq.csv')#(['mirna', 'seq']\n",
    "gene_seqdf=pd.read_csv('./datasets/train_dataset/gene_seq.csv')#'label', 'sequence'\n",
    "\n",
    "dataset_mirna=dataset['miRNA']\n",
    "dataset_gene=dataset['gene']\n",
    "dataset_label=dataset['label']\n",
    "gene_index=gene_seqdf['label'].values.tolist()\n",
    "gene_seq=gene_seqdf['sequence']\n",
    "mirna_index=mirna_seqdf['mirna'].values.tolist()\n",
    "mirna_seq=mirna_seqdf['seq']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 展示序列信息以及标签信息,以及关系对的标签信息"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gene_seq.head"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gene_index[0:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_label.head"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 数据预处理，特征提取和生成\n",
    "A:腺嘌呤、G:鸟嘌呤、T:胸腺嘧啶、C:胞嘧啶、U:脲嘧啶。\n",
    "\n",
    "其中T为DNA特有，U为RNA特有。\n",
    "\n",
    "配对规则：A=T(双氢键)、G=C(三氢键)，RNA为单链所以U无配对。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "key_set与key_set_T用于保存碱基子串计数,其中key为碱基子串(字符串)、value为计数\n",
    "两个循环分别用于初始化key_set与key_set_T,对其添加所有可能的碱基子串并初始化value为0\n",
    "'''\n",
    "# 用于碱基字串计数\n",
    "key_set={}\n",
    "key_set_T={}\n",
    "# itertools.product :可迭代对象输入的笛卡儿积 下面这一行可以生成每个元素3元组的可迭代对象\n",
    "for i in itertools.product('UCGA', repeat =3):#itertools.product('BCDEF', repeat = 2):\n",
    "    #print(i)\n",
    "    obj=''.join(i) #将三元组合并成一个字符串\n",
    "   # print(obj)\n",
    "    ky={'{}'.format(obj):0}\n",
    "    key_set.update(ky)\n",
    "for i in itertools.product('TCGA', repeat =3):#itertools.product('BCDEF', repeat = 2):\n",
    "    #print(i)\n",
    "    obj=''.join(i)\n",
    "   # print(obj)\n",
    "    ky={'{}'.format(obj):0}\n",
    "    key_set_T.update(ky)\n",
    "\n",
    "def clean_key_set(key_set):\n",
    "    '''\n",
    "    将字典所有的value设置为0\n",
    "    '''\n",
    "    for i,key in enumerate(key_set):\n",
    "    #print(i,key,key_set[key])\n",
    "      key_set[key]=0\n",
    "    return key_set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def return_features(n,seq):\n",
    "    '''\n",
    "    n为碱基子串的长度，seq为碱基序列\n",
    "    使用长度为n的滑动窗口选取碱基子串，并使用全局变量key_set记录字串数量并返回\n",
    "    '''\n",
    "    clean_key_set(key_set)\n",
    "    key=key_set\n",
    "    if '\\n' in seq:\n",
    "        seq=seq[0:-1]\n",
    "    for i in range(n,len(seq)+1-n):#忽略前n个碱基？\n",
    "        win=seq[i:i+n]#获取长度为n的碱基子串\n",
    "        #print(win)\n",
    "        ori=key_set['{}'.format(win)]#碱基子串计数增加\n",
    "        key_set['{}'.format(win)]=ori+1\n",
    "    return key_set\n",
    "def return_gene_features(n,seq):\n",
    "    '''\n",
    "    功能可参照上一个函数\n",
    "    '''\n",
    "    clean_key_set(key_set_T)\n",
    "    key=key_set_T\n",
    "    if '\\n' in seq:\n",
    "        seq=seq[0:-1]\n",
    "    for i in range(n,len(seq)+1-n):\n",
    "        win=seq[i:i+n]\n",
    "        #print(win)\n",
    "        ori=key_set_T['{}'.format(win)]\n",
    "        key_set_T['{}'.format(win)]=ori+1\n",
    "    return key_set_T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 使用拼接方法构建数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def construct_dataset(dataset_mirna,dataset_gene):\n",
    "    '''\n",
    "    将碱基子串的计数字典作为碱基序列的特征\n",
    "    将存在关系对的DNA与RNA的碱基序列特征直接连接起来作为训练用特征\n",
    "    '''\n",
    "    list_mirna_feature=[]\n",
    "    list_gene_feature=[]\n",
    "    for i in range(0,len(dataset_mirna)):# 遍历Train.csv中的RNA\n",
    "        try:\n",
    "            # 获取RNA\n",
    "            mirna=dataset_mirna[i]#获取当前Train.csv中RNA的名字\n",
    "            m_index=mirna_index.index(mirna)#根据名字获取到Train.csv中的RNA在mirna_seq.csv中的序号\n",
    "            mirna_f=return_features(3,mirna_seq[m_index])#输入当前RNA的碱基序列，并获取该序列的子串计数字典\n",
    "            # 获取DNA\n",
    "            gene=dataset_gene[i]#获取当前Train.csv中DNA的名字\n",
    "            g_index=gene_index.index(gene)#根据名字获取到Train.csv中的DNA在gene_seq.csv中的序号\n",
    "            gene_f=return_gene_features(3, gene_seq[g_index])#输入当前DNA的碱基序列，并获取该序列的子串计数字典\n",
    "\n",
    "            mirna_feature=mirna_f.copy()\n",
    "            gene_feature=gene_f.copy()\n",
    "            # 保存当前得到的子串计数\n",
    "            list_mirna_feature.append(mirna_feature)\n",
    "            list_gene_feature.append(gene_feature)\n",
    "        except:\n",
    "            mirna=dataset_mirna[i]\n",
    "            gene=dataset_gene[i]\n",
    "            print('error detected',i,mirna,gene)\n",
    "    # 合并得到的DNA，RNA碱基序列特征\n",
    "    lmpd=pd.DataFrame(list_mirna_feature)\n",
    "    lgpd=pd.DataFrame(list_gene_feature)\n",
    "    # 将对应关系的DNA，RNA序列特征直接拼接\n",
    "    X=pd.concat([lmpd,lgpd],axis=1)\n",
    "    return X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#标签换为数字\n",
    "Y=[]\n",
    "for i,label in enumerate(dataset_label):\n",
    "    if label =='Functional MTI':\n",
    "        Y.append(1)\n",
    "    else:\n",
    "        Y.append(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=construct_dataset(dataset_mirna,dataset_gene)\n",
    "#print(X)\n",
    "#lmpd.to_csv('gene_features.csv',index=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2模型训练&3.模型评估\n",
    "模型训练切分训练集调参，使用ACC和F1score作为评估标准"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, Y,test_size=0.8, random_state=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#切分训练集进行调参\n",
    "def train():\n",
    "  clf = RandomForestClassifier(n_estimators=30)\n",
    "  clf.fit(X_train,y_train)\n",
    "  y_p=clf.predict(X_test)\n",
    "\n",
    "  #acc = metrics.accuracy_score(y_test,y_p)\n",
    "  #print('RF_ACC',acc)\n",
    "  y_pb=clf.predict_proba(X_test)\n",
    "  #print(y_p)\n",
    "  f1score=metrics.f1_score(y_test, y_p)\n",
    "  print('RF_F1',f1score)\n",
    "  MCC=metrics.matthews_corrcoef(y_test, y_p)\n",
    "train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train():\n",
    "  clf = RandomForestClassifier(n_estimators=30)\n",
    "  clf.fit(X_train,y_train)\n",
    "  y_p=clf.predict(X_test)\n",
    "\n",
    "  #acc = metrics.accuracy_score(y_test,y_p)\n",
    "  #print('RF_ACC',acc)\n",
    "  y_pb=clf.predict_proba(X_test)\n",
    "  #print(y_p)\n",
    "  f1score=metrics.f1_score(y_test, y_p)\n",
    "  print('RF_F1',f1score)\n",
    "  MCC=metrics.matthews_corrcoef(y_test, y_p)\n",
    "train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 最终模型\n",
    "clf_final = RandomForestClassifier(n_estimators=30)\n",
    "clf_final.fit(X,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 存储模型与重新调用\n",
    "joblib.dump(clf_final,'./model/tran_model.m')\n",
    "clf_final = joblib.load('./model/tran_model.m')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.模型预测\n",
    "加载测试数据\n",
    "\n",
    "测试数据生成器\n",
    "\n",
    "预测结果\n",
    "\n",
    "结果展示\n",
    "\n",
    "生成提交文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#加载测试数据\n",
    "#test_filenames = os.listdir(\"./datasets\")\n",
    "df_predict=pd.read_csv('./datasets/test_dataset.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predict_mirna=df_predict['miRNA']\n",
    "predict_gene=df_predict['gene']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#测试数据生成器\n",
    "X_predict=construct_dataset(predict_mirna,predict_gene)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#预测结果\n",
    "final_result=clf_final.predict(X_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#结果展示\n",
    "print(final_result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成提交文件\n",
    "df_predict['results'] = final_result\n",
    "df_predict.to_csv('submission.csv',index=None)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "AI",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
